Multi-class Classification: A Coding Based Space Partitioning

In this work we address the problem of multi-class classification in machine learning. In particular, we consider the coding approach which converts a multi-class problem to several binary classification problems by mapping the binary labeled space into several partitioned binary labeled spaces through binary channel codes. By modeling this learning problem as a communication channel, these codes are meant to have error correcting capabilities and thus performance improvement in classification. However, we argue that conventional coding schemes designed for communication systems do not treat the space partitioning problem optimally, because they are heedless of the partitioning behavior of underlying binary classifiers. We discuss an approach which is optimal in terms of space partitioning and advise it as a powerful tool towards multi-class classification. We then review the LDA, a known method for multi-class case and compare its performance with the proposed method. We run the experiments on synthetic data in several scenarios and then on a real database for face identification.